Skip to content

Upgrade step and reset pipelines to proper pipelines with named functions - #66

Closed
amacati wants to merge 5 commits into
mainfrom
feat.named_pipelines
Closed

Upgrade step and reset pipelines to proper pipelines with named functions#66
amacati wants to merge 5 commits into
mainfrom
feat.named_pipelines

Conversation

@amacati

@amacati amacati commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Named, ordered simulation pipelines

Replaces the ad-hoc tuples of functions in sim.step_pipeline/sim.reset_pipeline with a Pipeline type: an ordered collection of named functions. Stages are addressed by name instead of by position, so you can splice in disturbances, randomization etc at a precise point without counting indices.

Stage names default to fn.__name__ and must be unique. Pass name= for anonymous callables (e.g. functools.partial). Iterating a pipeline yields the functions in order.

Usage

Pipelines are easier to inspect:

from crazyflow.sim import Sim
from crazyflow.sim.data import SimData

sim = Sim()

print(sim.step_pipeline)                # Pipeline(step_attitude_controller -> ... -> clip_floor_pos)
sim.step_pipeline.names                 # ordered stage names
"integration" in sim.step_pipeline      # True
sim.step_pipeline.index("integration")  # position of a stage
sim.step_pipeline["clip_floor_pos"]     # look up a stage's function by name
len(sim.step_pipeline)                  # number of stages

Inserting new functions also becomes easier with named access:

def disturbance(data: SimData) -> SimData:
    return data.replace(states=data.states.replace(vel=data.states.vel + 1e-5))

def monitor(data: SimData) -> SimData:
    return data

sim.step_pipeline.insert_before("integration", disturbance)
sim.step_pipeline.insert_after("integration", monitor)
sim.step_pipeline.prepend(monitor, name="pre_monitor")          # name= for a reused callable
sim.step_pipeline.replace("clip_floor_pos", lambda data: data)  # swap impl, keep name + position
sim.step_pipeline.remove("monitor")

We also allow some syntactic sugar to make common operations like appending easier

sim.step_pipeline = sim.step_pipeline + monitor  # Same as append with __name__
sim.step_pipeline.extend(some_other_pipeline)    # Adding multiple steps at once
for stage in sim.step_pipeline:                  # iterating yields the functions, in order
    ...

Pipelines also expose append / keys() / values() / items() and __repr__for nicer prints.

Notes:

  • + and construction accept functions or pipelines only. Use append / insert_* with name= for explicit names.
  • replace / remove raise on unknown names. Duplicate names raise on insert.

Closes #64

@amacati
amacati requested a review from ratheron June 15, 2026 10:43
@amacati amacati added the enhancement New feature or request label Jun 15, 2026

@ratheron ratheron left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very nice! I have two comments on the implementation

Comment thread examples/gradient.py
sim = Sim(control=Control.attitude, physics=Physics.first_principles, attitude_freq=50)
# Remove clipping floor function which kills gradients
sim.step_pipeline = sim.step_pipeline[:-1]
sim.step_pipeline.remove("clip_floor_pos")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The controller has clippings as well. Maybe we should add a note that either we remove the clipping or we start at a certain height.

Comment thread crazyflow/sim/sim.py

@jax.jit
def reset(data: SimData, default_data: SimData, mask: Array | None = None) -> SimData:
data = pytree_replace(data, default_data, mask) # Does not overwrite rng_key

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be a separate reset step, in case we want to remove it? Right now, the reset pipeline appears to be empty, even though the data is reset to the default data.

@amacati amacati mentioned this pull request Jun 16, 2026
@amacati

amacati commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator Author

Closed in favor of #68

@amacati amacati closed this Jun 16, 2026
@amacati
amacati deleted the feat.named_pipelines branch June 17, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No names in pipeline

2 participants